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READING  ABILITY  AND  OTHER  CORRELATES  OF  THE  SQT  WRITTEN  COMPONENT 

BRIEF 


Requirement: 

The  work  described  in  this  report  received  its  original  impetus  from  the  common 
complaint  that  the  Written  Component  (WC)  of  the  Skill  Qualification  Testing  (SQT) 
depends  too  much  upon  reading  ability,  despite  its  practical  orientation.  The  requirement 
statement  for  this  effort  specifies  research  to  determine  "the  relationship  between 
reading  skill  level  and  SQT  performance"  and  other  factors  related  to  success  on  the  SQT. 
The  objectives  which  guided  this  research  were: 

•  To  determine  the  correlation  between  reading  ability,  as 
measured  by  a  standard  test,  and  success  on  the  WC  of  selected 
SQTs. 

•  To  determine  the  correlation  between  the  WC  scores  and  demo¬ 
graphic  variables  (age,  education,  time  in  service). 

•  To  determine  the  correlation  between  the  WC  scores  and 
questionnaire  responses  pertaining  to  the  training  they  received, 
difficulty  in  understanding  the  Soldier's  Manual  and  questions  on 
the  WC,  and  satisfaction  with  their  Army  career. 

Procedure: 

A  computerized  data  base  was  established,  including  SQT  scores,  reading  test 
scores,  demographic  variables,  and  questionnaire  responses.  Correlational  techniques, 
including  factor  analysis  and  stepwise  regression,  were  used  to  determine  significant 
correlates  of  SQT  performance. 

Findings: 

•  Reading  ability  correlates  substantially  with  WC  scores. 

•  The  overall  £  =  .46,  with  particular  MOS  and  levels  varying 
around  that  figure.  But  much  of  that  correlation  may  be 
explained  by  general  ability,  since  reading  ability  also  correlates 
significantly  with  the  Hands-On  Component  (HOC)  (overall 
ir  =  .25). 

•  Soldiers  who  scored  low  on  the  WC  also  tended  to  report 
difficulty  in  understanding  their  Soldier's  Manuals  and  the  WC 
questions. 

•  Soldiers  who  got  their  Soldier's  Manuals  earlier  tended  to  score 
higher  on  the  WC. 

•  Differences  in  perceived  quality  of  training  were  not  appreciably 
correlated  with  WC  scores  in  the  sample  studied. 

•  SQT  scores  were  not  significantly  related  to  career  satisfaction. 

•  SQT  scores  were  not  significantly  related  to  age,  civilian  edu¬ 
cation,  or  time  in  service. 


v 


Utilization  of  Findings: 

The  Army  may  continue  to  use  the  multiple-choice  form  of  SQT  for  assessment  of 
job  skills  without  fear  that  the  scores  reflect  mostly  reading  ability,  rather  than  job 
performance.  However,  those  who  write  the  tests  should  try  to  simplify  the  wording  of 
questions,  because  the  scores  do  reflect  reading  ability  to  some  degree. 
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Chapter  1 

INTRODUCTION  AND  BACKGROUND 


hitroduction 

Skill  Qualification  Testing  (SQT)  was  first  used  in  the  Army  in  a  preliminary  form  in 
1976.  Testing  large  numbers  of  troops  for  record  in  several  Military  Occupational 
Specialties  (MOS)  began  in  the  1977  test  period,  hi  FY  1979,  a  total  of  456  SQT  tests  were 
developed  and  administered  for  various  MOS  and  levels. 

SQT  is  widely  acknowledged  as  a  significant  advance  over  the  previous  "knowledge 
based"  MOS  tests.  As  a  "criterion -referenced"  system,  it  is  concerned  with  measuring  the 
skills  actually  used  on  the  job,  applying  the  standards  needed  for  successful  performance. 
SQT  tests  consist  of  three  parts:  (1)  a  Written  Component  (WC)  which  involves  printed 
questions  and  multiple  choice,  machine  scored  answers,  (2)  a  Hands-On  Component  (HOC) 
which  involves  performance  with  the  actual  job  equipment,  and  (3)  a  Performance 
Certification  Component  (PCC)  which  involves  performance  in  an  environment  that 
cannot  be  readily  duplicated  in  a  test  situation  (e.g.,  rifle  marksmanship). 

However,  there  have  been  widespread  complaints  by  both  commanders  and  troops 
that  the  WC  still  depends  too  much  upon  reading  ability,  despite  the  practical  orientation 
of  the  tests.  Some  of  the  questions  still  have  a  "bookish"  or  "contrived"  quality  that  is 
characteristic  of  multiple  choice  exams.  One  consequence  of  the  criticism  has  been  to 
reduce  the  length  of  the  WC  and  its  relative  importance  in  the  scoring.  It  would  seem 
desirable  to  have  objective  evidence  whether  such  action  is  warranted,  or,  whether  even 
more  severe  restrictions  on  WC  items  should  be  imposed. 

On  the  other  hand,  WC  has  some  unique  advantages,  especially  efficiency  and  ease 
of  scoring.  Also,  some  soldier  tasks  can  be  simulated  very  well  with  the  WC  format  (e.g., 
map  reading).  Therefore,  it  is  desirable  to  find  out  the  advantages  and  disadvantages  of 
WC,  when  it  can  be  used  effectively,  and  how  to  do  it. 

The  present  report  is  an  attempt  to  assess  the  importance  of  reading  skill  and 
various  other  factors  on  WC  scores.  It  is  hoped  the  data  and  analysis  may  aid  in  decisions 
concerning  the  use  of  SQT. 


Hicks  *  administered  a  questionnaire  to  determine  soldiers'  opinions  about  training 
they  received  for  SQT  and  about  the  SQT  itself.  (That  study  was  an  antecedent  of  the 
present  study,  which  uses  the  same  questionnaire  for  some  variables.)  Hicks  reported  that 
about  "two  thirds  of  the  soldiers  expressed  satisfaction  with  the  Skill  Qualification 


lj.  A.  Hicks,  ID.  Skill  qualification  test  (SQT)  opinion  survey:  llB's,  lie's,  and 
llE's  (Research  Memorandum  78-8).  Alexandria,  Virginia:  US  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences,  March  1978. 


Testing  System.”  Of  particular  relevance  here  is  the  fact  that  most  soldiers  received 
their  Soldier's  Manuals  (SM)  less  than  six  months  before  testing,  contrary  to  the  goal 
according  to  the  SQT  handbook.  However,  almost  all  soldiers  in  the  sample  (90%) 
reported  receiving  their  SM  two  months  or  more  before  testing.  Almost  half  (43%)  of  the 
soldiers  who  read  their  SM  reported  some  difficulty  in  understanding  them.  Also,  almost 
half  (46%)  of  the  soldiers  reported  some  difficulty  in  understanding  questions  on  the  WC 
of  the  SQT. 

Another  questionnaire  survey  by  Yates^  assessed  the  impact  of  SQT  in  USAREUR 
for  infantry  units.  Her  results  pinpointed  some  administrative  and  training  problems 
including:  getting  the  Soldier's  Manuals  distributed  on  time  (six  months  before  testing), 
criticism  of  the  preparation  of  some  training  NCOs,  shortages  of  materials  and  equip¬ 
ment,  other  activities  interfering  with  training,  and  nontesting  of  some  eligible  personnel. 


^L.  G.  Yates.  The  estimated  impact  of  SQT  on  USAREUR  infantry  units:  Survey 
results  (Research  Problem  Review  79-9).  Alexandria,  Virginia:  US  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences,  June  1979. 
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Chapter  2 

DESIGN  AND  PROCEDURES 


The  analyses  in  this  report  involve  a  computerized  data  base  drawn  from  various 
sources,  related  on  the  basis  of  individuals  who  were  identified  by  the  last  four  digits  of 
their  social  security  numbers.  The  SQT  test  data  were  obtained  from  the  Individual 
Training  and  Evaluation  Directorate  (IT ED)  at  Fort  Eustis,  Virginia.  Fort  Hood's 
Education  Center  had  scores  on  a  standard  reading  test,  the  Metropolitan  Achievement 
Test.l  SIDPERS  (Standard  Ihstallation/Division  Personnel  System)  at  Fort  Hood  supplied 
demographic  data,  including  civilian  education  (in  numbers  of  years),  months  in  service, 
and  age.  A  questionnaire  about  the  SQT  and  preparation  for  it  was  administered  by  ARI 
to  soldiers.  Not  all  cases  in  the  data  base  included  information  from  all  three  sources,  so 
the  N  available  varies  with  the  type  of  analysis. 

This  report  analyzes  information  from  the  data  base  on  25  variables  of  interest,  as 
defined  in  Table  2-1.  The  general  analysis  is  a  correlation  matrix  relating  all  variables, 
based  upon  available  cases  that  had  scores  on  all  variables.  Since  the  data  cover  different 
MOS  and  levels,  all  SQT  scores  were  converted  to  percent  correct,  to  compensate  for 
different  lengths  of  the  various  tests. 2 

Of  primary  interest  are  the  intercorrelations  among  test  scores.  From  these  data, 
one  may  infer  the  importance  of  reading  skill,  as  defined  by  the  Metropolitan  Achieve¬ 
ment  Test,  for  scores  on  the  Written  Component  (WC).  One  may  also  estimate,  very 
crudely,  how  much  of  the  correlation  may  be  explained  by  "general  ability,"  since  the  test 
data  include  scores  on  the  Hands-On  Component  (HOC),  which  requires  no  reading. 

The  demographic  variables,  in  conjunction  with  test  scores,  indicate  the  influence  of 
formal  education,  time  in  service,  and  age  upon  reading  skill  and  SQT  scores.  These  data 
indicate  whether  career  soldiers  are  much  like  recruits  in  the  tested  skills  and  education, 
or  distinctly  different. 

The  questionnaires  encompass  so  many  variables  that  patterns  are  difficult  to 
discern,  and  objectivity  of  apparent  trends  is  suspect.  Factor  analysis  was  used  to 
indicate  which  items  involved  related  issues,  and  which  items  were  independent  of  each 
other. 


*W.  N.  Durost,  H.  H.  Bixler,  J.  W.  Wrightstone,  G.  A.  Prescott,  &  L  H.  Balow. 
Metropolitan  achievement  tests,  Form  G.  Harcourt,  Brace,  Jovanovich,  1971. 

2ft  is  recognized  that  the  conversion  to  percentages  does  not  fully  neutralize 
differences  in  the  various  tests  involved.  Therefore,  the  correlations  tend  to  underesti¬ 
mate  somewhat  the  correlations  that  would  be  obtained  by  considering  each  test 
separately. 
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TABLE  2-1.  Definition  of  Variables 


Questionnaire  items  (see  appendix  for  exact  wording) 


No.  of 


Symbol 

Content  Distinct  Values 

Polarity* 

Def  of  Scale 

Q3 

Validity  of  SQT 

2 

- 

Yes-No 

Q4 

Fairness  of  admin  of  SQT 

2 

- 

Yes-No 

Q5 

When  SN  received 

8 

+ 

From  0-6+  months 

Q6 

When  SM  received 

8 

+ 

(as  above) 

Q7 

When  SQT  tng  started 

8 

+ 

(as  above) 

Q8 

Difficulty  understanding 

5 

- 

Extreme-0,  & 

SM 

didn't  read  SM 

Q9 

Difficulty  understanding 

5 

- 

(as  above) 

WC  questions 

Q10 

Independent  study  f/SQT 

4 

- 

Extreme-0 

Qll 

Satisfied  w/HOC  tng 

6 

- 

Ext  sat-ext  dissat 

Q12 

Satisfied  w/WC  tng 

6 

- 

(as  above) 

Q13 

Supervisor  qua!  f/SQT  tng 

6 

- 

Ext  qual-ext  disqual 

Q14 

Like  SQT  tng 

6 

- 

Liked  ext-disl  ext 

Q15 

Like  MOS 

6 

- 

(as  above) 

Q16 

Career  satisfaction 

6 

- 

Ext  sat-ext  dissat 

Q17 

SQT  important  f/career 

6 

- 

Ext  imp-ext  unimp 

Q18 

(see  appendix,  note) 

Q19 

How  you  did  on  SQT 

3 

+ 

<60%  to  >80% 

Demographic  variables 

CIVED 

Civilian  education 

18 

+ 

in  years,  0-17 

SERMO 

Months  in  service 

136 

+ 

AGE 

Age  in  years 

25 

+ 

Test  Scores 

WPCT 

SC,  %  scoring  units  go 

+ 

0-100% 

HPCT 

HOC,  %  scoring  units  go 

+ 

0-100% 

TOTPCT 

WC+HOC,  %  scoring  units  go 

+ 

0-100% 

WRDKNOW  Test  1,  Met  Ach  Test 

+ 

Grade  lvl,  0-9.9  max 

RDCOMP  Test  2,  Met  Ach  Test 

+ 

(as  above) 

RDLEVL 

Test  1+2,  Met  Ach  Test 

+ 

(as  above) 

a"Polarity"  refers  to  whether  the  answer  choices  are  sequenced  in  an  increasing  (+)  or 
decreasing  scale  (-),  which  will  determine  whether  correlations  will  be  positive  or  negative. 


The  correlations  of  questionnaire  responses  with  test  data  may  indicate  relative 
importance  of  various  training  and  motivational  factors  for  SQT  scores.  These  data  may 
also  indicate  whether  attitudes  about  SQT  are  related  to  performance  on  the  tests,  or 
independent.  A  stepwise  regression  analysis  was  used  to  assess  the  relative  importance  of 
questionnaire  variables  for  WC  scores.  The  stepwise  regression  analysis  was  also  run  on 
data  for  each  MOS  separately  to  see  whether  the  same  pattern  of  results  was  present  in 
all  MOS. 

An  item  analysis  was  conducted  to  see  how  each  question  on  the  WC  was  related  to 
reading  level  and  total  score.  One  index  of  interest  is  reversals  (when  people  who  miss 
the  item  score  higher  on  the  reading  test  or  WC  total  score).  Reversals,  or  even  weak 
discriminators,  are  less  useful  as  items.  Bad  items  might  also  be  indicated  by  most 
people  missing  the  question.  The  most  discriminating  items  are  also  of  interest. 


Chapter  3 

RESULTS  AND  ANALYSES 


A  correlation  matrix  for  all  25  variables  is  presented  in  Table  3-1.  The  matrix  is 
sectioned  according  to  the  three  kinds  of  variables  involved  (test  scores,  demographic 
data,  and  responses  to  questionnaire  items).  A  correlation  must  be  at  least  +.12  to  be 
statistically  significant*  with  N  =  255  (j>  <.5,  two-tail  test)  and  anything  less  will  be 
considered  as  having  negligible  practical  significance  also.  Therefore,  r_  =  +.12  will  be  the 
minimum  criterion  in  the  discussion  that  follows. 

Test  Scores 


Correlations  among  test  scores  are  presented  in  the  right  hand  section  (tail)  of  the 
matrix.  The  most  important  correlations  are  between  Written  Percent  Correct  (WPCT) 
and  reading  test  scores  (.44,  .40,  and  .46,  for  word  knowledge,  reading  comprehension  and 
reading  level,  respectively).  This  indicates  a  substantial  relationship  between  reading 
ability  and  WPCT. 

However,  part  of  these  correlations  may  be  explained  by  a  "g"  factor  ("general 
ability"  or  "general  intelligence")  that  relates  to  both  written  and  hands-on  tests,  rather 
than  a  special  reading  factor.  Hands-on  Percent  Correct  (HPCT)  also  correlates 
somewhat  with  the  reading  scores  (.24,  .24,  and  .25),  and  the  hands-on  tests  involve  no 
reading.  Also,  the  lower  correlations  with  HPCT  may  be  attributable  partly  to  lower 
reliability  of  HPCT,  because  of  shorter  tests  and  much  higher  average  scores.  The  mean 
HPCT  was  91  (vs.  61  for  WPCT),  indicating  little  room  for  improvement  on  the  hands-on 
test. 


Die  correlation  between  WPCT  and  HPCT  is  .23,  which  is  modest  considering  that 
they  are  both  parts  of  the  SQT.  A  somewhat  larger  correlation  (£=  .30  was  obtained  in  a 
more  inclusive  sample  (N  =  424  when  we  include  soldiers  who  did  not  take  the  question¬ 
naire),  but  the  correlation  is  still  considered  modest.  (No  other  correlation  for  the  larger 
sample  differed  appreciably  from  those  presented  in  Table  3-1.)  HPCT  correlated  about 
as  much  with  the  reading  tests  as  with  WPCT,  indicating  some  sort  of  "g"  factor 
underlying  all  three  tests. 

Demographic  Variables 

The  highest  correlation  among  these  variables,  years  of  civilian  education  (CIVED), 
months  in  service  (SERMO),  and  age  in  years  (AGE)  indicates  that  people  who  have  been  in 
the  Army  longer  also  tend  to  be  older  (r  =  .77)  which  is  hardly  surprising.  There  is  also 
some  tendency  for  more  formal  education  among  older  =  .22)  and  more  senior  (£  =  .18) 
soldiers.  These  correlations  with  education  may  reflect  education  while  in  service,  or  a 
tendency  to  eliminate  those  with  less  education  from  the  Army,  or  a  combination  of  the 
two. 


lW.  H.  Beyer  (ed.).  Handbook  of  tables  for  probability  and  statistics,  2nd  edition. 
Cleveland,  Ohio:  Chemical  Rubber  Company,  1966,  pp  392-393. 


TABLE  3-1.  Correlation  Matrix8  (N  *  255)^ 


I 

■ 

r’  . 

i 

► 


3 

I 

■ 


23 

Q4 

25 

26 

21 

28  22 

210  211 

Q12 

212 

Q14  i 

212 

Q16 

211 

Q18 

Q19 

Q4 

22 

Q5 

-15 

-14 

Q6 

-10 

-21 

53 

Q7 

-08 

-11 

41 

40 

Q8 

-02 

-04 

07 

08 

-03 

Q9 

-19 

-20 

11 

13 

17 

33 

Q10 

03 

-05  ■ 

-07 

-09 

-10 

30 

15 

Qll 

08 

07  • 

-11 

-08 

02 

10 

09 

21 

Q12 

16 

18  • 

-07 

-07 

-13 

06 

-12 

19 

31 

Q13 

04 

07  • 

-02 

-07 

-08 

07  ■ 

-02 

06 

09 

24 

Q14 

20 

06  ■ 

-04 

-04 

09 

07 

03 

22 

30 

20 

19 

Q15 

09 

11 

02 

-03 

-12 

00 

03 

20 

08 

17 

09 

29 

Q16 

10 

09 

03 

03 

-00 

17 

05 

28 

17 

14 

09 

37 

57 

Q17 

16 

12  - 

-06 

-19 

-11 

-04 

01 

29 

10 

10 

18 

34 

35 

45 

Q18 

-03 

-01  - 

-04 

02 

-07 

08  - 

-08 

-02  ■ 

-02 

07 

-02 

03 

01 

11 

03 

Q19 

-16 

-13 

04 

19 

00 

13 

17 

-12  ■ 

-05 

-16 

02 

-07 

-17 

-04 

-28 

-02 

C1VED 

-02 

-01 

10 

13 

03 

-03  ■ 

-03 

-07  - 

-10 

-00 

-01 

-05 

-02 

-01 

-07 

-01 

-01 

SERMO 

-04 

-04 

09 

08 

25 

-04 

-05 

-21 

00 

-13 

00 

-02 

-01 

-08 

-16 

-10 

-00 

AGE 

01 

-00 

04 

00 

15 

-02  • 

-11 

-14  - 

-02 

-12 

01 

-03 

02 

-02 

-10 

-11 

-06 

WPCT 

-09 

-15 

08 

26 

11 

17 

24 

07  • 

-06 

-14 

04 

08 

-02 

13 

-11 

-00 

40 

HPCT 

-11 

-17 

02 

09 

-02 

-10 

00 

-03  ■ 

-32 

-01 

-01 

-08 

-01 

-02 

-08 

11 

20 

TOTPCT 

-08 

-15 

07 

25 

05 

14 

21 

-07  - 

-11 

-13 

04 

07 

-01 

13 

-11 

02 

41 

WRDKNOW 

00 

-09 

06 

13 

-03 

18 

13 

-04 

02 

-05 

-03 

06 

10 

14 

01 

04 

14 

RDCOMP 

01 

-09  - 

-03 

10 

-05 

09 

13 

-04  • 

-00 

-08 

-01 

-01 

05 

12 

-04 

-01 

17 

RDLEVL 

-01 

-09 

02 

12 

-04 

16 

17 

-04  • 

-01 

-07 

-03 

03 

10 

13 

-02 

00 

19 

CIVED 

SERMO 

AGE 

WPCT 

HPCT 

TOTPCT 

WRDKNOW  RDCOMP 

SERMO 

18 

AGE 

22 

77 

WPCT 

-03 

05 

-01 

HPCT 

09 

08 

02 

23 

TOTPCT 

-01 

03 

-03 

98 

38 

WRDKNOW 

05 

08 

03 

44 

24 

47 

RDCOMP 

12 

17 

12 

41 

24 

43 

67 

RDLEVL 

10 

15 

09 

46 

25 

49 

88 

92 

aDecimal  points  omitted. 
^Breakdown  of  N: 


2 

level 

3 

4 

1  IB 

99 

21 

11 

11C 

20 

8 

1 

1  ID 

1 

1 

1 

HE 

58 

34 

0 

7 


Inc  Variables  and  Test  Scores 


SQT  scores  are  not  significantly  related  to  years  of  education,  time  in  the  Army,  or 
age.  Senior  service  members  seem  to  have  a  slight  advantage  on  Reading  Comprehension 
(r_  =  .17)  but  not  on  Word  Knowledge.  Education  seems  to  have  little  or  no  relationship 
with  reading  scores,  nor  does  age. 


There  were  many  significant  correlations  between  questionnaire  responses.  There¬ 
fore,  a  factor  analysis  was  conducted  for  a  larger  sample,  including  all  cases  available  in 
the  data  base  (N  =  529).  The  analysis  was  done  using  a  varimax  rotation  with  Program 
BMDP4M.2  The  correlation  matrix  is  presented  in  Table  3-2,  and  the  rotated  factor 
loadings  in  Table  3-3.  The  four  factors  obtained  explain  about  half  the  questionnaire 
variance  (47.94%). 

Factor  1  might  be  called  "career  motivation."  It  seems  to  involve  satisfaction  with 
Army  career  and  MOS,  with  a  priority  on  SQT  training  to  further  that  career.  Factor  2  is 
perhaps  the  easiest  one  to  define,  involving  advance  preparation  given  for  SQT.  All  three 
items  begin  "How  far  in  advance  of  the  test  date  did...";  no  other  item  included  such  a 
statement,  or  its  equivalent.  Factor  3  might  be  called  "satisfaction  with  SQT  training" 
that  was  administered  (Qll-14).  It  also  involved  a  tendency  to  feel  that  SQT  was  fair 
(Q3).  Factor  4  may  be  called  "comprehension"  of  printed  material  (Q8, 9).  It  also 
involved  a  tendency  to  feel  that  the  SQT  is  valid  (Q4).  Factor  4  concerns  the  active  role 
of  the  individual  soldier  in  preparing  for  SQT,  while  Factor  3  focuses  upon  the  training 
administered  to  the  servicemember. 


estionnaire  Items  and  Dem 


ic  Data 


Generally  the  questionnaire  responses  were  not  significantly  related  to  age,  educa¬ 
tion,  or  length  of  service.  However,  the  senior  people  (SERMO)  seemed  to  take  the  SQT 
more  seriously;  they  studied  more  (Q10,  r_  =  -.21),  thought  it  affected  their  Army  career 
(Q17,  r  = -.16),  and  were  more  satisfied  with  their  training  for  the  WC  (Q12,  r  =  -.13). 
They  also  thought  training  started  earlier  (Q7,  r  =  .25),  probably  because  senior  people  are 
responsible  for  giving  the  training,  so  their  part  does  begin  earlier.  The  same  pattern  of 
correlations  appears  with  AGE  (which  is  highly  correlated  with  seniority)  but  the 
correlations  are  much  smaller,  so  only  two  of  them  are  significant  (Q7,  r  =.15;  Q10, 
r  =  .14). 

The  only  correlation  with  civilian  education  that  reaches  our  criterion  of  signifi¬ 
cance  is  Q6  (r  =  .13).  There  seems  to  be  no  obvious  explanation  for  this  marginally 
significant  result. 


2W.  J.  Dixon  &  M.  B.  Brown.  BMDP-79,  Biomedial  computer  programs  (P- Series). 
Berkeley:  University  of  California  Press,  1979. 


TABLE  3-2.  Correlation  Matrix8  Used  For  Factor  Analysis  Of 
Questionnaire  Responses  (N  ■  529)^ 


22 

§4 

21 

21 

2Z. 

21 

2£ 

Q10 

2 11 

Q4 

22 

Q5 

-12 

-14 

Q6 

-15 

-12 

55 

Q7 

-07 

-11 

51 

41 

Q8 

-06 

-08 

08 

06 

01 

Q9 

-22 

-20 

13 

12 

11 

39 

Q10 

04 

-01 

-13 

-15 

-08 

24 

14 

Qll 

16 

13 

-03 

-04 

01 

05 

-01 

16 

Q12 

25 

16 

-08 

-07 

-06 

05 

-15 

17 

37 

Q13 

16 

17 

-04 

-06 

-08 

03 

-05 

07 

22 

Q14 

18 

07 

02 

-03 

09 

09 

-01 

25 

30 

Q15 

06 

12 

-02 

-04 

-09 

03 

08 

24 

11 

Q16 

05 

07 

01 

01 

-03 

13 

09 

27 

16 

Q17 

12 

09 

-03 

-14 

-08 

04 

03 

31 

12 

Q18 

-05 

01 

-04 

01 

-02 

04 

-03 

13 

-02 

Q19 

-17 

-11 

08 

16 

02 

16 

22 

-10 

-10 

Q12  Q13  Q14  Q15  Q16  Q17  Q18  Q19 


30 

29 

25 

16 

17 

35 

16 

19 

40 

57 

09 

22 

35 

38 

48 

07 

04 

11 

06 

18 

-18 

02 

-05 

-08 

-01 

“Decimal  points  omitted, 
b Breakdown  of  N: 


2 

level 

3 

4 

11B 

210 

68 

33 

11C 

42 

16 

3 

UD 

1 

1 

1 

11E 

98 

54 

2 
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TABLE  3-3.  Sorted  Rotated  Factor  Loadings  (Pattern)* 


Factor  1  Factor  2 


Factor  3 


Factor  4 


Q16 

.81 

Q15 

.80 

Q17 

.75 

Q14 

.52 

.45 

Q5 

.84 

Q7 

.79 

Q6 

.78 

Q12 

.76 

Qll 

.70 

Q13 

.51 

Q8 

.76 

Q9 

.74 

Q10 

.46 

Q3 

.50 

Q4 

-.38 

Z  Variance 
Explained 

14.00 

12.18 

11.88 

9.88 

aThe  above  factor  loading  matrix  has  been  rearranged  so  that  the  columns 
appear  in  decreasing  order  of  variance  explained  by  factors.  The  rows  have  been 
rearranged  so  that  for  each  successive  factor,  loadings  greater  than  0.30  appear 
first.  Loadings  less  than  0.35  have  been  omitted. 


(The  questionnaire  factors  presented  in  Table  3-3  may  be  related  to  the  various  test 
scores  as  follows.) 

Factor  1  ("career  motivation")  is  virtually  unrelated  to  test  scores.  The  only 
significant  correlations  are  with  Q16,  and  those  are  negative:  people  who  are  satisfied 
with  their  Army  career  tend  to  do  slightly  worse  on  WPCT  (£  =  .13)  and  WRDKNOW 
(r  =  .14).  (These  correlations  are  of  marginal  significance.)  Apparently,  seriousness  about 
Army  career  has  no  appreciable  effect  upon  SQT  performance. 

From  Factor  2  ("How  far  in  advance...”),  Q6  is  related  to  WPCT  (£  =  .26),  indicating 
that  having  the  SM  in  time  to  study  is  important.  Question  6  also  seems  to  be  correlated 
with  WRDKNOW  (£  =  .13)  but  there  seems  to  be  no  apparent  explanation. 

From  Factor  3  ("Satisfaction  with  SQT  training"),  Q12  correlates  with  WPCT 
(r  =  -.14),  indicating  that  satisfaction  with  WC  training  is  related  marginally  to  perform¬ 
ance  on  the  corresponding  test.  Question  11  correlated  with  HPCT  (r^  =  —.32),  indicating 
that  satisfaction  with  HOC  training  is  related  substantially  to  performance  on  the 
corresponding  test.  No  other  correlation  of  Factor  3  variables  with  test  scores  is 
significant.  Apparently,  differences  in  perceived  quality  of  training  are  not  crucial  for 
SQT. 


All  Factor  4  ("comprehension")  variables  are  significantly  related  to  WPCT  (Q9, 
£  =  .24;  Q8,  r  =  .17;  Q4,  £  =  -.15).  People  who  say  they  can  understand  the  SM  and  WC 
perform  better.  People  who  think  that  SQT  is  valid  (Q4)  are  those  who  score  higher  on 
WPCT  (r  =  -.15)  and  also  on  HPCT  (r  =  .17).  People  who  say  they  understand  the  SM  and 
WC  questions  also  do  better  on  the  reading  tests  (r  =  .16  and  .17  with  RDLEVL). 

Question  19  ("How  well  do  you  think  you  did  on  SQT")  correlates  highly  with  WPCT 
(£  =  .40),  with  HPCT  (£  =  .20)  and  with  TOTPCT  (£=.41).  Clearly,  people  can  estimate 
pretty  well  how  they  did  immediately  after  taking  the  test.  However,  these  correlations 
are  considered  to  be  of  little  practical  significance  for  success  on  these  tests,  and  perhaps 
misleading. 

Stepwise  Regression 

A  stepwise  regression  analysis  was  conducted  using  Program  BMOP2R^  for  the 
questionnaire  responses,  with  WPCT  as  the  dependent  variable.  Question  19  was  omitted 
because  it  seemed  misleading  (as  noted  above)  and  because  it  was  correlated  substantially 
with  the  dependent  variable.  The  analysis  was  done  first  for  all  MOS,  then  for  each  MOS 
separately.  Table  3-4  summarizes  these  analyses. 

The  significant  variables  in  these  analyses  generally  confirm  the  significance  of  the 
correlations  discussed  above.  Question  6  (getting  their  SM  in  time  to  study  for  SQT) 
emerges  as  the  most  important  item  for  performance  on  WC.  Understanding  the  SM  and 
the  WC  (Q8,  9)  is  also  generally  important.  Independent  study  (Q10)  also  is  significant  for 
WC.  (The  corresponding  correlation  was  not  significant  for  the  smaller  sample  presented 
in  Table  3-1.) 


^Dixon  &  Brown,  op  cit. 
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TABLE  3-4.  Stepwise  Regression 


Minimal  acceptable  F  to  enter...  4.0 


SUMMARY 

TABLE: 

All  MOS  (N  -  534) 

Step 

Variable 

Multiple 

Increase 

No. 

Entered 

Removed 

R 

RSQ 

in  RSQ 

1 

Q6 

.28 

.08 

.08 

2 

Q9 

.31 

.10 

.02 

3 

Q10 

.33 

.11 

.01 

4 

Q8 

.35 

.12 

.01 

E(WPCT)=“36. 136+2 

!.005(Q6)+2.664(Q9)-2.639(Q10) 

SUMMARY 

TABLE: 

11B  (N  *  317) 

Step 

Variable 

Multiple 

Increase 

No. 

Entered 

Removed 

R 

RSQ 

in  RSQ 

1 

Q6 

.25 

.06 

.06 

2 

Q9 

.30 

.09 

.02 

3 

Q5 

.33 

.11 

.02 

4 

Q10 

.35 

.13 

.02 

5 

Q8 

.37 

.14 

.01 

E(WPCT)= 

=40.380-1 

•554(Q5)+2. 7441 (Q6)+2. 893 (Q8)- 

-2.998(Q10) 

SUMMARY 

TABLE: 

11C  (N  *  62) 

Step 

Variable 

Multiple 

Increase 

No. 

Entered 

Removed 

R 

RSQ 

in  RSQ 

1 

Q6 

.41 

.17 

.17 

2 

Q9 

.49 

.24 

.07 

3 

Q10 

.55 

.31 

.07 

4 

Q16 

.60 

.36 

.05 

E(WPCT)*21 . 789+2 

.335(Q6)+6.631(Q9)=6.388(Q10)+2.294«U6) 

SUMMARY 

TABLE: 

11D  (N  =  2) 

INSUFFICIENT  CASES  FOR  ANALYSIS 

SUMMARY 

TABLE: 

11E  (N  -  153) 

Step 

Variable 

Multiple 

Increase 

No. 

Entered 

Removed 

R 

RSQ 

in  RSQ 

1 

Q6 

.27 

.07 

.07 

E(WPC)-44. 657+1. 956(Q6) 

Getting  the  Soldier's  Notice  in  time  to  study  (Q5)  was  significant  only  for  11B. 
Satisfaction  with  Army  career  (Q16)  appeared  as  a  significant  step  for  11C  only;  this 
seems  strange  since  those  who  do  well  on  WC  would  appear  to  be  more  dissatisfied  with 
their  Army  career.  (The  correlation  was  positive,  p  =  .32.)  It  may  be  a  chance  result:  the 
5%  confidence  interval  of  t_  for  N  =  62  is  about  +.25,  and  the  1%  confidence  interval, 
about  +.33.^ 

Satisfaction  with  WC  training  (Q12)  did  not  appear  in  the  stepwise  regression, 
although  the  correlation  was  marginally  significant  in  the  correlation  matrix  (Table  3-1) 
discussed  above.  No  other  inconsistencies  were  noted  between  the  stepwise  regression 
and  the  correlational  analysis  discussed  previously. 


A  Model  Relating  the  Significant  Variables 

The  significant  correlations  involving  WPCT  seem  to  suggest  a  model  relating  the 
variables  that  were  crucial  for  success  on  the  written  tests.  The  model  traces  the 
dissemination  of  task  information  from  an  authoritative  source  to  the  soldier's  correct 
responses  on  the  test. 

First,  the  soldier  must  get  his  Soldier's  Manual  in  time  to  study  it  before  the  test 
(Q6).  It  may  also  help  if  he  gets  his  Soldier's  Notice  telling  him  what  to  study  (Q5)  in  the 
SM.  It  helps  if  he  studies  on  his  own  (Q10)  and  he  must  be  able  to  understand  what  he 
reads  (Q8  and  RDLEVL).  He  must  also  be  able  to  understand  the  written  questions  when 
he  takes  the  test  (Q9  and  RDLEVL). 

It  does  not  seem  to  make  much  difference  whether  the  soldier  likes  his  career  (Q16) 
or  his  MOS  (Q15).  Nor  does  it  matter  much  whether  he  thinks  SQT  is  important  for  his 
career  (Q17)  or  fair  (Q3). 

It  did  not  make  much  difference  whether  the  SQT  training  the  soldier  received 
seemed  good  or  bad  (Qll,  Q12,  Q13,  Q14).  (However,  there  is  a  slight  correlation  when 
his  judgments  are  specific  to  WC  training.)  This  trend  would  seem  to  minimize  the 
importance  of  differences  in  the  unit's  formal  training  programs.  However,  the  training 
might  seem  more  critical  if  a  greater  range  of  programs  were  included.  Also,  these 
judgments  may  be  more  global  and  subjective  and,  hence,  unreliable,  than  some  other 
questions  (like  whether  he  could  understand  the  questions  on  WC). 

It  does  seem  critical  to  insure  that  the  Soldier's  Manuals  be  available,  and  that  they 
should  include  all  task  information  needed  to  pass  the  test  expressed  in  a  comprehensible 
form.  The  WC  should  also  be  comprehensible,  and  the  necessary  reading  limited  to  what 
is  needed  for  the  job. 


^Beyer,  op.  cit.,  pp  393-393. 


For  each  question  in  the  1182  WC,  the  average  reading  level  of  those  who  were 
correct  was  compared  with  the  average  for  those  who  were  wrong.  Those  comparisons 
would  indicate  which  items  were  especially  dependent  upon  reading  ability,  and  which 
were  not.  The  correct  and  incorrect  groups  on  each  item  were  also  compared  on  total  WC 
score,  to  determine  which  items  were  most  discriminating.  Significance  was  determined 
by  t^- tests,  using  a  pooled  estimate  of  the  variance.^ 

The  11B2  test  analysis  was  selected  because  that  was  the  test  with  the  largest  N. 
Comparable  analyses  were  also  conducted  in  other  WCs,  and  results  were  similar,  except 
for  irregularities  and  lower  significance  levels  because  of  fewer  data. 

The  vast  majority  of  items  were  statistically  significant  discriminators  on  total  WC 
score  and,  to  a  lesser  extent,  on  reading  level.  On  total  WC  scores,  157  questions  were 
significant  discriminators,  7  were  positive  but  not  significant,  and  there  was  1  reversal. 
On  reading  level,  116  questions  were  significant  discriminators,  41  were  positive  but  not 
significant,  and  there  were  8  reversals.  There  were  another  four  questions  that  no  one 
missed.  The  greater  significance  on  total  WC  score  was  due  to  the  greater  N  (401  vs.  197) 
as  well  as  dissimilarities  between  the  tests. 

None  of  the  reversals  on  either  criterion  was  statistically  significant.  The  one 
reversal  on  total  WC  score  (Q23-3)  was  also  a  reversal  on  reading  level.  Most  people 
(70%)  also  missed  that  item.  That  was  the  only  item  that  the  data  even  suggested  might 
be  scored  wrong  or  be  generally  misleading.  There  were  only  15  items  that  were  missed 
by  more  than  half  of  the  people. 

The  test  content  was  examined  to  determine  what  characteristics  were  associated 
with  discrimination  on  either  criterion.  No  such  characteristics  were  readily  apparent. 
Nor  are  there  many  items  that  are  highly  discriminating  on  one  criterion  but  not  on  the 
other.  The  form  and  content  of  the  questions  will  be  analyzed  in  another  report. 


^Dixon  &  Brown,  op.  cit.,  pp  172-173. 
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Chapter  4 

SUMMARY  AND  CONCLUSIONS 


Correlation*  Among  Teats 

SQT  scores  for  infantrymen  and  armor  crewmen  (11B,  11C,  and  HE)  were  correlated 
with  scores  on  a  standard  reading  test  and  with  questionnaire  responses  to  determine  what 
factors  were  associated  with  success  on  SQT.  In  particular,  the  data  were  analyzed  to 
evaluate  the  common  complaint  that  the  multiple-choice  questions  (WC)  measured 
reading  ability  that  was  unrelated  to  performance  of  their  jobs. 

Performance  on  WC  is  moderately  correlated  with  reading  ability,  but  some  of  that 
correlation  may  be  explained  by  the  fact  that  any  test  scores  are  apt  to  correlate  to  some 
degree.  Even  the  SQT  performance  test  (HOC)  correlated  significantly  with  the  reading 
test,  but  not  so  highly  as  the  multiple-choice  part  did. 


Demographic  Data 

Test  scores  generally  did  not  correlate  significantly  with  demographic  data  (civilian 
education,  age,  or  time  in  service).  The  one  exception  was  a  slight  tendency  for  those 
longer  in  the  service  to  score  higher  on  one  section  of  the  reading  test  (r_  =  .17). 

Neither  did  these  demographic  data  correlate  generally  with  questionnaire 
responses.  However,  there  was  a  slight  pattern  indicating  that  senior  soldiers  tended  to 
take  SQT  a  little  more  seriously. 


Questionnaire  Data  and  SQT  Scores 

A  factor  analysis  of  questionnaire  responses  revealed  four  factors: 

1.  Career  motivation  (investment  in  an  Army  career).  This 
seems  virtually  unrelated  to  test  scores. 

2.  Advance  preparation  for  SQT.  Each  of  these  questions 
involved  how  far  in  advance  of  SQT  did  some  training  event 
occur.  The  only  thing  that  made  an  appreciable  difference 
was  getting  the  Soldier's  Manual  in  time  to  study  for  the  WC. 

3.  Satisfaction  with  SQT  training.  Soldiers  who  were  satisfied 
with  their  training  for  WC  or  HOC  tended  to  do  a  little 
better  on  that  particular  component  of  the  SQT.  But 
generally,  their  SQT  scores  were  unrelated  to  their  opinions 
on  the  quality  of  training  they  received. 

4.  Comprehension.  Soldiers  who  said  they  could  understand  WC 
questions  or  their  SM  tended  to  score  better  on  all  tests. 


Item  analysis  of  the  11B2  WC  indicated  consistently  good  quality  of  test  item,  i.e., 
those  who  got  an  item  right  also  tended  to  have  a  high  total  score.  Almost  all  items  (95%) 
were  significant  discriminators,  and  only  one  item  showed  a  reversal.  This  indicates  that 
no  question  was  consistently  misleading. 

Comchiafan 

The  criticism  that  SQT  scores  reflect  mostly  reading  ability  rather  than  job  skills  is 
not  valid.  However,  test  developers  should  continue  trying  to  simplify  the  wording  of 
questions,  because  the  scores  do  reflect  reading  ability  to  some  degree. 
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SURVEY  FORM 


DATA  REQUIRED  BY  THE  PRIVACY  ACT  OF  1974 


TITLE:  SQT  Opinion  Survey 

PRESCRIBING  DIRECTIVE:  AR  70-1 
AUTHORITY  10  USC  Sec  4503 

PURPOSE(s):  The  data  collected  with  the  attached  forms  are  to  be  used 

for  research  purposes  only. 

This  is  an  experimental  personnel  data  collection  form 
developed  by  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  pursuant  to  its  research 
missions  as  prescribed  in  AR  70-1.  When  identifiers  (name 
or  Social  Security  Number)  are  requested  they  are  to  be 
used  for  administrative  and  statistical  control  purposes 
only.  Full  confidentiality  of  the  responses  will  be  main¬ 
tained  in  the  processing  of  these  data. 

Your  participation  in  this  research  is  strictly  voluntary. 
Individuals  are  encouraged  to  provide  complete  and  accu¬ 
rate  information  in  the  interests  of  the  research,  but  there 
will  be  no  effect  on  individuals  for  not  providing  all  or  any 
part  of  the  information. 


SQT  Opinion  Survey 
INSTRUCTIONS 


The  purpose  of  this  questionnaire  is  to  obtain  your  opinions  about  the  Skill 
Qualification  Test  (SQT)  which  you  recently  took  and  about  the  training  which  you 
received  to  prepare  you  for  the  test.  Your  responses  are  very  important  because 
they  will  be  used  to  help  improve  the  SQT  and  SQT  training  procedures.  Please  be 
as  accurate  and  honest  as  you  can  when  answering  all  of  the  items  in  the 
questionnaire.  Your  responses  to  this  questionnaire  are  being  collected  for  research 
purposes  and  the  confidentiality  of  your  responses  will  be  maintained.  Please 
answer  all  the  questions  on  the  answer  sheet  provided.  Do  not  write  in  the 
questionnaire  booklet.  It  is  important  that  you  fill  out  the  answer  sheet  very 
carefully. 

1.  Last  Name:  (Enter  on  the  answer  sheet) 

2.  Social  Security  Number:  (Enter  on  the  answer  sheet) 

3.  Do  you  feel  that  your  performance  on  the  SQT  will  be  a  good  indication  of 
your  proficiency  in  your  MOS?  (Choose  one) 

A.  Yes 

B.  No 

4.  Do  you  feel  that  the  SQT  was  administered  fairly?  (Choose  one) 

A.  Yes 

B.  No 

5.  How  far  in  advance  of  the  test  date  did  you  receive  your  SQT  notice  telling 
you  which  tasks  from  the  Soldier's  Manual  would  be  on  the  SQT?  (Choose  one) 

A.  I  did  not  receive  an  SQT  notice 

B.  Less  than  one  month 

C.  One  month  or  more,  but  less  than  two  months 

D.  Two  months  or  more,  but  less  than  three  months 

E.  Three  months  or  more,  but  less  than  four  months 

F.  Four  months  or  more,  but  less  than  five  months 

G.  Five  months  or  more,  but  less  than  six  months 

H.  Six  months  or  more 

6.  How  far  in  advance  of  the  test  date  did  you  receive  your  Soldier's  Manual? 
(Choose  one) 

A.  I  did  not  receive  an  SQT  notice 

B.  Less  than  one  month 

C.  One  month  or  more,  but  less  than  two  months 

D.  Two  months  or  more,  but  less  than  three  months 

E.  Three  months  or  more,  but  less  than  four  months 

F.  Four  months  or  more,  but  less  than  five  months 

G.  Five  months  or  more,  but  less  than  six  months 

H.  Six  months  or  more 


7.  How  far  in  advance  of  the  test  date  did  your  unit  begin  training  for  the  SQT? 
(Choose  one) 

A.  My  unit  did  not  conduct  any  SQT  training 
6.  Less  than  one  month 

C.  One  month  or  more,  but  less  than  two  months 

D.  Two  months  or  more,  but  less  than  three  months 

E.  Three  months  or  more,  but  less  than  four  months 

F.  Four  months  or  more,  but  less  than  five  months 

G.  Five  months  or  more,  but  less  than  six  months 

H.  Six  months  or  more 

8.  Did  you  have  any  difficulty  understanding  the  Soldier's  Manual  while  preparing 
for  the  SQT?  (Choose  one) 

A.  Extreme  difficulty 

B.  Moderate  difficulty 

C.  Slight  difficulty 

D.  No  difficulty 

E.  I  did  not  read  the  Soldier's  Manual 

9.  Did  you  have  any  difficulty  understanding  the  questions  in  the  Written 
Component  of  the  SQT?  (Choose  one) 

A.  Extreme  difficulty 

B.  Moderate  difficulty 

C.  Slight  difficulty 

D.  No  difficulty 

10.  Did  you  do  any  studying  or  your  own  to  prepare  yourself  to  take  the  SQT? 
(Choose  one) 

A.  An  extreme  amount 

B.  A  moderate  amount 

C.  A  slight  amount 

D.  None  at  all 

11.  How  satisfied  or  dissatisfied  are  you  with  the  training  you  received  for  the 
Hands-On  Component  of  the  SQT?  (Choose  one) 

A.  Extremely  satisfied 

B.  Moderately  satisfied 

C.  Slightly  satisfied 

D.  Slightly  dissatisfied 

E.  Moderately  dissatisfied 

F.  Extremely  dissatisfied 

12.  How  satisfied  or  dissatisfied  are  you  with  the  training  which  you  received  for 
the  Written  Component  of  the  SQT?  (Choose  one) 

A.  Extremely  satisfied 

B.  Moderately  satisfied 

C.  Slightly  satisfied 

D.  Slightly  dissatisfied 

E.  Moderately  dissatisfied 

F.  Extremely  dissatisfied 
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How  qualified  or  unqualified  was  your  immediate  supervisor  to  tram  you  for 
the  SQT?  (Choose  one) 

A.  Extremely  qualified 

B.  Moderately  qualified 

C.  Slightly  qualified 

D.  Slightly  unqualified 

E.  Moderately  unqualified 

F.  Extremely  unqualified 

How  much  did  you  like  or  dislike  training  for  the  SQT?  (Choose  one) 

A.  Liked  extremely 

B.  Liked  moderately 

C.  Liked  slightly 

D.  Disliked  slightly 

E.  Disliked  moderately 

F.  Disliked  extremely 

How  much  do  you  like  or  dislike  your  MOS?  (Choose  one) 

A.  Like  extremely 

B.  Like  moderately 

C.  Like  slightly 

D.  Dislike  slightly 

E.  Dislike  moderately 

F.  Dislike  extremely 

Overall,  how  satisfied  or  dissatisfied  are  you  with  your  Army  career? 

(Choose  one) 

A.  Extremely  satisfied 

B.  Moderately  satisfied 

C.  Slightly  satisfied 

D.  Slightly  dissatisfied 

E.  Moderately  dissatisfied 

F.  Extremely  dissatisfied 

How  important  or  unimportant  do  you  fell  that  your  SQT  performance  will  be 
to  your  Army  career?  (Choose  one) 

A.  Extremely  important 

B.  Moderately  important 

C.  Slightly  important 

D.  Slightly  unimportant 

E.  Moderately  unimportant 

F.  Extremely  unimportant 
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.  Do  you  intend  to  reenlist?  (Choose  one) 

A.  Yes 

B.  No  {See  Note) 

C.  Undecided 

19*  Overall,  how  well  do  you  think  you  did  on  the  SQT?  (Choose  one) 

A.  Passed  less  than  60 %  of  the  tasks 

B.  Passed  60-79%  of  the  tasks  (Qualified  in  MOS) 

C.  Passed  80-100%  of  the  tasks  (Promotable  in  MOS) 


PLEASE  COMPLETE  QUESTIONS  20  THRU  23  WHICH  ARE  ATTACHED  TO  THE 
ANSWER  SHEET. 


♦The  answer  choices  to  this  item  were  inadvertantly  sequenced,  so  they  do  not 
correspond  to  the  underlying  dimension  of  inclination  to  re-enlist.  Therefore, 
correlations  with  this  item  should  be  be  interpreted  in  terms  of  Army  career 
intentions. 


